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INTRODUCTIOH 

In inid-1970 the Univcrsitx of California at Santa Barbara (UCSB) 
initiated the devclopnicnt of a training program for supervising teachers 
who work with student teachers in the schools. Training activities focused 
on four skill areas judged to be critical in the performance of field super- 
visors; namely^ observing, analyzing, prescribing, and counseling. These 
areas were treated as sequential and systemically related rather than separate 
and discrete. 

During 1970-71, IS elementary school and 14 secondary school supervising 
teachers participated in the first round of training.* A set of 10 tests 
served to provide information on whether the training achieved the desired 
results and to generate formative feedback for appropriate program modifica- 
tion.** 

Test scores indicated that the training program did yield acceptable 
results in the skill areas of observing, prescribing, and counseling.*** 
Only in the area of analyzing student teachers* behavior did the training 
fail to deliver a satisfactory pattern of desired changes in the performance 
of the trainees. 



*The U.S. Office of Education supported the first year's effort through 
Grant No. OEG-0-70-1902. 

**The following instruments were administered on a pre and post basis: 
(1) Observation-Specific; (2) Observation-Judgmental Language; (3) Observation- 
Student Teacher Problems; (4) Observation-General; (5) Prescription; (6) 
Analysis-Written; (7) Counseling Theory; (8) Counseling Practice; (9) Situa- 
tional (10) Meaning Measurement Inventory. 

***See Norman J. Boyan, et . al. , A Program for Training Supervising 
Teachers in the Induction of New Professionals : Final Report for 1970-71 , 
U.S. Office of Education Grant No. OEG-0-70-1902. Santa Barbara, California: 
University of California at Santa Barbara, November 1971. 



/imklysin of test d^ta mA verbal feedback from panicipatit$t led the UCSB 
itaff to modify substantially the supervision training program in 1971-^72. 
The four'^stage conceptualization of supervision used in 1970*71 (observationi 
analysiSi prescription, counseling) gave way to an eight-step Supervision 
Process Modeli which guided the selection of objectives, activities, and 
evaluation procedures in the second round of training. ♦ The SPM included 
the following steps: (1) id'^ntification of an area of concern; (2) when 
appropriate, identification of a criterion of desired performance; (3) 
selection of a measuring or observing instrument; (4) observation of the 
selected areas of interest; (5) analysis of the obtained data; (6) identifica- 
tion of needed changes and alternative procedures to accomplish these 
changes; (7) provision of feedback to or with the student teacher; (8) 
selection of instructional strategies for bringing about the desired changes, 
with provision for the student teacher to implement the selected strategy 
and for recycling the entire eight-step process. Emphasis in the relationship 
between supervising teacher and student teacher shifted from a supervisor- 
dominated base to achievement of a cooperative, collegial arrangement. Trans- 
fer of supervision skills from supervising teachers to student teachers in 
order to enhance the capability of the latter to assess their own teaching 
took its place as one of the specific objectives of the training program. 

Trainees in 1971-72 included 26 "new'' supervising teachers (11 from the 
elementary school level and 15 from the secondary school level) and 10 
"continuing" supervising teachers who had participated in the 1970-71 train- 
ing program. The ''new" and "continuing" groups were treated separately in 



*The U.S. Office of Education supported the second year's effort 
through continuation of its Grant No- OEG- 0-70- 1902. 
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the asdessnent o( their performances on seven tests, five adialnlstered on a 
pre and post basis, two administered on a post basis only.* 

Evaluation results revealed that the training delivered nearly all of 
the objectives of the program at levels judged satisfactory by the training 
staff.** Both "new** and "continuing** supervisors demonstrated appropriate 
use of the revised supervision model. Like their counterparts of the previous 
year, the trainees increased their ability to distinguish between observa- 
tional and judgmental statements, they substantially increased the number of 
behavioral observations made when viewing teaching behavior, and they reduced 
their tendency to report judgments without supporting observations. Trainees 
also revelaed enlarged capacity to analyze the results of observational data 
by specifying summary data and identifying patterns in the data. In addition 
they were able to specify substantially more techniques for achieving desired 
behavioral changes on the part of student teachers. After training, the 
supervising teachers demonstrated sa':isfactory attainment of theoretical 
knowledge about the process of behavioral counseling and they performed more 
of the counseling behaviors established as training objectives. 

The 1971-72 training program included a separate experimental study of 
the effects of "trained" vs. "untrained" supervising teachers on the ability 
of student teachers to learn and demonstrate selected components of the 
Supervision Process Model collected into a Self -Evaluation Model, The study 



*The instruments administered on a pre and post basis included (1) 
Observation-Specific Test; (2) Observation-General Test, (3) Observation- 
Judgmental Language Test; (4) Analysis Test; (5) Situational Test, The 
instruments administered on a post basis only included (1) Counseling Theory 
Test and (2) Field Test, 

**See Norman J. Boyan, et, al. , A Program for Training Supervising 
Teachers in the Induction of New Professionals: Final Report for 1971-72 , 
U.S. Office of Education Grant No. OEG-0-70-1902. Santa Barbara, California: 
University of California at Santa Barbara, August 1972. 



^ yielded conclusive evidence that the performance of student teicherd super- 
vised by •'trained'* supervising teachers was superior on target behaviors to 
the performance of student teachers working with '•untrained'* supervising 
ti^achers.* 

DEVELOPMENT OF THE INSTRUCTIONAL SUPERVISION 
TRAINING PROGRAM 

Encouraged by the results of the first two years of effort, the U.S. 
Office of Education contracted with UCSB to prepare a "hands-off" instruc- 
tional package for improving supervision skills in Teacher Corps projects. 
Team leaders were identified as the primary tarp/^ts for training; however, the 
scope of work also recognized supervisors in teacher education programs 
generally and supervisors in local education agencies as potential benefi- 
ciaries. 

The major task of the UCSB staff was to convert the previous year's 
training program into a set of materials which Teacher Corps projects could 
use without the participation or intervention of the developers. Accordingly, 
the staff launched a systematic development effort to prepare, try out, and 
revise a self-contained package which would include all of the directions and 
all of the instructional units required by a user to conduct, on his own, a 
successful training program for supervisors. 

The development work involved (1) refining the prevailing view of super- 
vision to which training would address itself, (2) designing specifications 
for the instructional package, (3) preparing the preliminary units of 
instruction, (4) pilot testing and revising the preliminary units of instruc- 
tion, (5) field testing the instructional units, and (6) revising the units 



*See Glenn S. Pate, "Training Student Teachers to Perform Specific 
Self -Evaluation Behaviors", Ph.D. dissertation. University of California, 
Santa Barbara, March, 1973, 
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m the basis of fldld test results. 

During the late Summer of 1972, the staff created a preliminary set of 
directions and training materials in a fom Judged to be specifically 
oriented to supervisors in Teacher Corps projects. Preparation of the 
preliminary form drew on (1) detailed analysis of the results achieved in 
the 1971-72 training program, (2) further review of pertinent literature 
(3) the expertise of the UCSB staff and consultants, and (4) commentary from 
Teacher Corps personnel about the conditions of instructional supervision 
especially applicable to their situations.* The preparatory activities led 
to the generation of a fresh set of terminal objectives, enabling objectives, 
training activities and materials, and associated evaluation instruments and 
procedures . 

The Instructional Supervision Process 

A new name. Instructional Supervision Process (ISP), was coined as a 
replacement for the old title of Supervision Process Model to signify the 
introduction of several specific alterations in the conceptualization of 
supervision which guided the establishment of training objectives and 
construction of training materials for 1972-73 • A summary of changes made 
from 1971-72 to 1972-73 in the guiding "models" appears in Table 1.** 

Assumptions invoked by the staff in building a model of the Instructional 
Supervision Process included (1) that instructional supervision must focus on 
a teacher's instructional concerns, particularly with respect to the effect 



*For details on Teacher Corps relationships see Norman J- Boyan, et, al, . 
The Instructional Supervision Training Program: Final Report for 1972-73 , 
U-S, Office of Education Grant No, 0EG-0-72-1155(715) . Santa Barbara, 
California: Graduate School of Education, University of California, September 
1973. 

**The use of the term, "model", refers here to graphic illustration of 
steps incorporated into the Supervision Process Model and the Instructional 
Supervision Process, 



TABLE 1 

A SUMMARY OF CHANGES MADE IN THE CONCEPTUALIATION 
OF INSTRUCTIONAL SUPERVISION 



Supervision Process Model 
(1971-72) 

1. Process initiated by supervisor 
identification of teaching 
problem perceived in the student 
teacher. 

2. Supervisor concerns emphasized 
in all steps of the Process. 

3. Observational data presented to 
student teacher by supervisor, 
with supervisor's analysis of 
the data. 

4. Supervisor suggests hypotheses 
and change strategies to deal 
with instructional problem of 
student teacher. 

5. Supervisor serves as the 
authority in identification 
and solution of problem. 



6. Method of interaction between 
supervisor and supervisee 
dep^^ndent on abilities acquired 
in other settings (e.g., special 
training in counseling or inter- 
personal relations in extra- 
supervisory situations). 



Instructional Supervision Process 
(1972-73) 

1. Process initiated by supervisee 
or supervisor identification 
of perceived teaching problem. 



Supervisee concerns emphasized 
in all steps of the process. 

Data display of observation 
presented to supervisee by super- 
visor, and supervisee guided in 
his own analysis of the data. 

With supervisory guidance, the 
supervisee formulates hypotheses 
and change strategies to deal 
with instructional problems. 

Supervisor guides the supervisee 
in the development of skills 
relevant to the identification and 
analysis of instructional problems 
and to proposing possible solutions. 

Supervisor employs techniques of 
communication and interaction 
specifically selected for their 
ability to facilitate supervisee 
growth in confidence and efficacy 
in the supervising relationship. 
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of the teachcr^s behavior on pupils; (2) that a sy;;tci&atic and objective 
approach to instructional suporvlsion will aid supervisor and supervisee^ 
together » to identify find resolve the latter* s specific instructional 
problems; (3) th?it instructional supervision operate: best when it takes 
the forw of a nonthreatcning, supp^^i ^vo, and hcJoing relationship between 
supervisor and supervisee; and, (4) that instructional supervision operates 
iBost effectively when the strengths of a systctnatic problem- solving approach 
are combined with a nonthreatening, supportive, and helping relationship 
between supervisee and supervisor. Through emphasis on systematic method, 
ISP attempts to free the supervisor and supervisee from primary dependence 
upon Intuitive, "one- shot" efforts for the improveiiient of instruction. 
Further, ISP calls upon the supervisor and supervisee to learn to use the 
process together in a joint effort to resolve instructional problems. 

The process itself starts when the beginning teacher identifies with 
his instructional supervisor an area of teaching concern. Immediately, 
the supervisor helps the supervisee to define the problem (Step 1). Together, 
the two parties agree upon specific teaching/learning behaviors that may 
be related to the supervisee's problem (Step 2), and establish a method 
by which these target behaviors may be observed in the classroom (Step 3). 
The supervisor observes the supervisee in a classroom setting and sytem- 
atically records observations using personally selected, constructed, or 
adapted instruments (Step 4) . The supervisor analyzes the data from his 
observation (Step 5) and determines the target behaviors which, because of 
their potential effect upon pupils should be changed or maintained (Step 6) . 
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Ifithout threat 9 the supervisor prdvides to the supervisee a clear sumsiary 
of the data froa the clasaroon observation and aids him in its analysis 
(Step 7). The supervisee then conpares the target behaviors vhich are 
revealed by his own analysis with his previous perception of those behaviors 
and identifies behavior changes which taay resolve the problra (Step 8). 

Following the post*obscrvation conference^ the supervisee attempts to 
introduce the desired behavior changes in his/her classroom and may be 
provided with training to do so. The supervisee and supervisor can then 
recycle the process to verify the change attempt. When the instructional 
problem is resolved » the supervisor t^akcs every effort to sec that the 
supervisee fully understands the reasons for its resolution. 
Terminal Objectives of the ISTP 

Successful movement through and use of the Instructional Supervision 
Process require a supervisor to employ a set of skills which the UCSB 
staff translated into the terminal objectives of the Instructional Super- 
vision Training Program listed below: 

1. (Given a pre-observation conference setting,) together with the 
supervisee, to identify and define behaviorally an area of concern. 

2. (Given a pre-observation conference setting,) to determine whether the 
establishment of a base rate or the setting of criteria is appropriate. 

3. (Given a pre-observation conference setting in which the establishment 
of criteria is appropriate,) together with the supervisee (a) to 
identify criteria for the successful performance of a terminal goal; 
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(b) ifhdn ctieh critetia are tidt reatisti^aliy within the {tfesent Cdpabi]^ 
itii^f df 4 supervisee^ to idettHty criteria tot the successful perf^or* 
mace of intenediate goals* 

4. (Civeti a statement of a supervisee's area of concern J to select « 
construct I or adapt an instrument for systematically recording the 
occurrence of teaching/learning behaviors which nay be related to the 
area of concern* 

5. (Given an instrunent for systematically recording the occurrence of 
specific teaching/learning behaviors^) to utilize the instrument to 
make non'^evalualive observations in a classroom sett Inc. 

6. (Given the data resulting from the systematic observation of teaching/ 
learning behaviors in a classroom setting J to analyze the data by 
performing the following operations: 

a. Construct a data display; 

b. Make summary statements; 

c. Where criteria of successful performance have been established » 
determine if the criteria have been met; 

d. Identify patterns of behavior; 

a. Make comparisons between observations. 

7. (Given the results of an analysis of data from a systematic observa;:ioni) 
to Identify: 

a. Supervisee behaviors that are positive in terms of their possible 
pupil effects and should be maintained; 

b. Supervisee behaviors that arr negative in terms of their possible 
pupil effects and shoald be changed; 

c. Alternate strategies for producing changes in supervisee behavior. 

8. (Given a post-obsorvation conference setting and given the data from 
the observation and analysis^ to perform the following operations: 

a. Review with the supervisee the agreed upon area of concern « its 
behavioral definition* the decision to obtain a base rate or the 
criterion that was sot* and the observation instrument; 

b. Guide the supervisee in an analysis of the observ<?d data which will 
approximate the complete analysis obtained by the supervisor; 

c. Assist the supervisee in identifying behaviors which are positive in 
terms of their possible pupil effects and should be maintained; 

d. Assist the supervisee in identifying behaviors which are negative in 
terms of their possible pupil effects i^nd should be changed; 

e. Formulate with the supervisee strategies! for achieving the desired 
changes; 

£. Determine with the supervisee if additional cycles of the ISP are 
necessary. 

9. (Given a pre- or post-observation conference settings) establish clear 
communication between supervisor and supervisee by utilising coiAjnunica- 
tion skills* such as paraphrasing* perception checking* asking clarifying 
questions, offering information* and active* attentive listening* 
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10. C61V6II i pte^or post ^dbservat Ion tonterence settings atid <in under* 

itAndiiijt of the distinction betireen **freeing^* and *'binding" dtatcnentdj 
to de^mstrate the use of freeing and binding responses in order to 
mcourage the ^owth of autonomous behavior in the supervisee. 

Tril nlng Mater iai s 

After speclfy'ing teralnal objectives^ the staff addressed itself to the 
pteparation of training activities and materials which would deliver the 
dpsiri*d competencies. A *'hands*off^ instructional package was designed^ 
consisting of the following pieces: (1) a Training_Coordinator*s Guide . 
(2) a set of Readings » (3) an associated set of Handout s » (4) five video* 
tapes » and (S) four audio tapes* The prelininary package incorporated seven 
different instructional units involving a variety of training modes. 

Preparation of the Training Coordinator's Guide was crucial to insure 
the likelihood that an on-site trainer who had no prior exposure to or 
experience with ISTP would find all of the instructions and references needed 
to conduct training sessions. The Guide provided the local coordinator 
with »tcp*by«step directions for carrying out all activities, including 
instructions to participants, assignments to be made, materials to be 
used in each session. Tho training coordinator also received a supply of 
Handouts for distribution to participants at designated times throughout the 
training period. The Handouts contained answers to exercises, role playing 
instructions, and other information for delivery to participants on a 
specified schedule. 

Tho training program itself contoincd throe types of instructional units: 
information acquisition, role playing, and practicum. Information acquisi- 
tion units stressed knowledge about the various stages and steps of the ISP, 
with instruction presented through a multi-media array of printed materials, 
video tapes, audio tapes, transparencies, and written exercises. Basic Infor- 
mation about the ISP and its constituent skills appeared in a book of 
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^ Readings which each participant received at the beginning of the training 
period. Videotapes, audiotapes, and transparencies demonstrated the use 
of various skills, presorted sample instructional situations, and illustrated 
concepts. A written exercise typically followed the presentation of each 
major concept to enhance the occurrence of comprehension. Participants 
usually completed the exercises on a self-paced basis, with immediate feed- 
back available by way of an answer sheet. Role playing units emphasized 
development of skill in application of ISP as a whole and acquisition of 
analysis competencies in particular, through the use of trio and simulation 
exercises. Trio exercises required individual participants to interact 
with colleagues in a role playing design as they gained understanding of 
central concepts and as they practiced use of specific skills. Simulation 
exercises called upon participants to employ previously acquired concepts 
and skills in situations which closely resemble supervisory conferences, with 
each member of a trio rotating through the roles of supervisor, supervisee, 
and observer. Stimuli for both forms of role playing exercises came fron 

video tapes, audio tapes, or written situational statements and dialogues. 
Practicum units focused on the development of synthesis and evaluation com- 
petencies pertinent to the use of ISP, by providing participants oppor- 
tunities to employ the Process and its constituent skills in actual field 
settings. A practicum unit required the trainee to conduct a "real" 
supervisory conference with a "real" supervisee and to record the proceedings 
on audio tape. Critique of the performance came to the participant from 
his fellow trio members during a follow up training period. 

Training Format , A 36-hour workshop arrangement was selected as the 
time frame and format for conducting training. The UCSB staff chose the 
workshop mode to permit the scheduling of training activities under a 
variety of conditions, such as concentration of longer sessions in a one or 
O two-week period or distribution of shorter sessions on a weekly basis over a 
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wmth or DorOi doponding on tho length of sessions. Sessions themselves 
incltided large group presentations for basic instruction^ seminars for 
discussions amopg participants » and opportunities for extensive practice in 
the use of ISP and its constituent competencies through exercises in the 
trio mode. T! j trios not only enabled trainees to receive immediate feed- 
back on thair performances from fellow participants but also squared with 
the stress ir ISTP on trainees* acquiring supervisory competencies while 
engaged in personal interaction with others. 
Pilot Testing 

During preparation of the preliminary instructional units and Training 
Coordinator's Guide ^ a group of supervising teachers in the Santa Barbara 
area critiqued the content and sequence of the materials. Most of the 
CTitiquers had participated in the 1971-72 UCSB training program for super- 
vising teachers. These "reality demons" contributed enormously in making 
the preliminary instructional units ready for pilot testing. 

Approximately 40 hours of pilot testing helped to determine the extent 
to which the content and procedures aided or impeded attainment of the 
specified objectives. The preliminary set of instructional units was 
administered under the directions contained in the Training Coordinator's 
Guide, to two groups of three participants each. The training coordinator 
for each group was unfamiliar with ISTP prior to conducting training 
sessions. Each of the coordinators was an experienced teacher and super- 
visor. The trainees in the pilot test were all experienced public school 
teachers. At the time of pilot testing, none of the participants was super- 
vising a student or intern teacher. 

The opportunity to monitor closely the administration of the instructional 
package by a local group of trainers to a local group of trainees led to 



incorporation of numerous revisions in the training program, including the 
following: 

(1) provision of more visual aids (e.g., transparencies) for use by 
trainers with trainees; 

(2) provision of more detailed instructions to training coordinator 
and participants for practice (role playing) cycles; 

(5) stipulation of sharper and more precise time estimates for com- 
pleting individual training exercises; and, 

(4) substantial editorial revision of written materials for improved 
grammar, punctuation, and syntax. 

Field Testing the ISTP 

Of the set of Teacher Corps project directors contacted by letter or 
telephone to ascertain potential interest in field testing the preliminary 
version of ISTP, 15 responded positively. Each received a detailed 
description of training and testing requirements and a sampler of the training 
package. After review of the requirements and sample materials, 12 directors 
agreed to engage in the field test. Three withdrew from participation, on 
the grounds of local scheduling and administrative problems, prior to 
actual engagement; one never completed the test battery. As a result, eight 
Teacher Corps sites took part in the full field test during the Spring of 
1973.* The UCSB staff also arranged for the participation of a local group 
to insure availability of evaluative data for revising the preliminary 
version of ISTP in the event that logistical or diplomatic problems inter- 
fered with the collection of useful information from the eight distant sites. 
The training coordinator and participants at each locale received by mid- 
January, 1973, all of the instructional materials required for conducting the 



*The sites were (1) Texas Southern University, (2) Winston-Salem State 
University, (3) Alabama ASM University, (4) University State College at 
Buffalo, (5) Idaho State University, (6) University of California at Santa 
Cruz, (7) Weber State College, (8) Memphis State University. 
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training sessions and all of the evaluation instruments required for admin- 
istering the testing program. 

Coordinators at each site were requested to administer the pretest prior 
to the initiation of training sessions and to return the test protocols to 
UCSB for scoring. At those sites where coordinators honored the request, 
pretests took place between January 15 and January 26, 1973. The proposed 
schedule of events called for training to proceed between the end of January 
and the middle of April and for posttests to be completed by the end of April. 
Only four of the Teacher Corps sites and the local (Santa Barbara) group 
adhered to the desired time table; the other four Teacher Corps sites exper- 
ienced a number of problems which delayed their completion of all training 
and testing until the end of May. The UCSB staff scored test protocols as 
materials arrived from the field but pooled all test results in conducting 
its subsequent analyses, irrespective of the particular site in which and 
irrespective of the varying conditions under which training had occurred. 

FIELD TEST RESULTS 
An adaptation of a Solomon Four-Group research design (see Table 2) 
served to ascertain the extent to which the preliminary version of ISTP 
achieved the terminal objectives set for trainees and to provide estimates 
of testing effects. The expectation was that subjects who received training 
would score significantly higher on the exercises comprising a specially 
constructed Instructional Supervision Test Battery than subjects who did not 
receive training. The Battery consisted of six sections: a two-part 
Comprehension Exercise, a two-part Observation Exercise, an Analysis Exer- 
cise, and a Participant Performance Record (PPR) . Parallel forms of the 
exercises were used for pre and posttests; the PPR was used as a posttest 
only, and only with the two experimental groups. 

- 13 - 



TABLE 2 

Treatment and Testing Conditions 
for Experimental and Control Groups 



Group 


Treatment and Testing Conditions 


Pre-Tested Received Training Post-Tested 


Experimental I 


X XX 


Experimental II 


X X 


Control I 


X X 


Control II 


X 



Subjects 

Subjects included Teacher Corps team leaders (and counterparts who super- 
vised interns) drawn from eight volunteer sites and a group of volunteer 
s\ui)ervising teachers located in public schools close to UCSB. The manner in 
with subjects were recruited does not permit any claim for representatii;eness 
among all Teacher Corps personnel or ail public school supervising teachers. 

Moreover, the design employed for analysis of field test results 
assumes homogeneity of subject groups, preferably achieved by random assign- 
ment, and ideally calls for approximately equal numbers in all cells* Given 
the constraints which prevailed across all training sites, neither random 
assignment of subjects nor equal distribution of subjects according to 
training and testing conditions was attainable. 

Homogeneity of groujs . The UCSB staff conducted two separate analyses 
to ascertain the extent of homogeneity which did prevail across the two 
experimental and two control groups. The first considered some 16 personal 
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characteristics and previous experiences which appeared potentially salient 
as contributors to variability in training effects.* The second considered 
pretest scores on five sections of the Instructional Supei-vision Test Battery 
for subjects who did and who did not receive training. 

Chi-square analyses performed on the personal characteristics revealed 
seven statistically significant differences in the composition of the four 
groups. However, when the entire experimental group was compared with the 
entire control group, a statistically significant difference showed itself 
on only two characteristics; type of credential possessed by the trainees 
and grade level taught by the beginning teachers supervised by the trainees. 
The experimental group, considered as a whole, included a larger proportion 
of subjects who had earned one or more credentials authorizing service in 
the public schools than did the control group. Also, approximately 20 
percent of the experimental group was supervising, at the time of training, 
beginning teachers who were teaching at the secondary school level while 
the control group did not include any subjects who were supervising 
secondary novitiates. 

There is little logical reason to expect lack of homogeneity on the 
characteristic of a beginning teacher *s grade level assignment to affect 
materially the results of training provided his or her supervisor. In 
addition, evaluation results from previous years in the UCSB supervisor 



*A Background Information Sheet was used to collect information about 
subjects on the following characteristics and experiences: (1) sex (2) age 
(3) education (4) teaching credential (5) ethnic background (6) local 
education agency environment (7) occupational role (8) teaching experience 
(9) supervision experience (10) counseling experience (11) training in inter- 
personal relations (12) training in systematic classroom observation (13) 
training in supervision (14) training in writing behavioral objectives 
(15) ethiiic background of supervisee (16) grade level taught by supervisee. 
Actually the data collected for the 16 variables enabled comparisons to be 
made on 18 counts, because the variable entitled "occupational role" was 
subdivided into three roles: teacher, supervisor, administi^ator. 
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training programs had not unearthed any systematic association between 
subjects' abilities to demonstrate target skills after training and the 
grade levels at which the subjects and their student teachers woi^ked. With 
respect to the difference between the proportions of credentialed and non- 
credentialed trainees in the experimental and control groups, some discom- 
fort about the assumption of homogeneity does enter, on the grounds that a 
higher incidence of professional preparation may have provided the exper- 
imental group a potential edge in benefitting from training in supervision. 

The staff also compared pretest scores on the Instructional Supervision 
Test Battery for the pertinent experimental and control groups (Experimental 
I and Control I) in order to ascertain the extent to which these subjects 
entered the treatment with essentially similar competencies. Statistically 
significant differences between the pretested experimental and control 
groups appeared in none of the 26 summary measures included in the Compre- 
hension Exercise, the Observation Exercises, and the Analysis Exercise. 

All told, then, direct examination of the composition of experimental 
and control groups and assessment of pretest scores for subsets of the two 
groups provide reasonable solace with respect to the assumption of homo- 
geneity. The assumption does not, however, enjoy full support. Results 
reported for the field test should, therefore, be interpreted in the light of 
the limitation unavoidably introduced because the UCSB staff could not 
secure random assignment of subjects across all nine training sites. 

Equal distribution of subjects in experimental and control groups . 
Participating sites received requests to establish control groups of approxi- 
nately the same size as the groups receiving training via ISTP and to 
assign approximately equal numbers of trainees and controls to sit for both 
pretest and posttest and for the posttest only. 
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Ihrfortunately, the respective sites did not or were not able to respond 
faithfully to requests pertinent to the distribution of subjects by treat- 
ment and testing conditions. Two of the nine sites did not provide any 
control subjects; two provided information only for individuals who sat for 
both the pretest and posttest; and two provided only posttest protocols. 
Table 3 summarizes the distribution of experimental and control subjects by 
type of test involvement, 

TABLE 3 

Distribution of Experimental and Control Group 
Subjects fay Type of Test Involvement 

Group 

Control Experimental Total 
Pretest and Posttest 19 61 80 

Pretest Only 12 15 27 

Total 31 76 107 

To account for the influence of unequal cell sizes in the analyses of 
variance performed to assess training and testing effects, the sum of 
squares was estimated by using the unweighted means procedure recommended 
by Winer.* 
Treatment 

Administration of the preliminary version of ISTP by on-site training 
coordinators was the treatment provided to experimental subjects. The 
UCSB staff did not intervene directly in any training activities other than 
distributing full sets of ISTP materials and directions for testing. 
Coordinators did not have previous experience with and had not received 
previous instruction in the use of the materials from the developers. 

The only guaranteed common element in the treatment was the substance 
contained in the Training Coordinator's Guide ^ Readings > Handouts , audio 
tapes* video tapes, and test instructions delivered to each training site. 



♦B.J. Winer, Statistical Principles in Experimental Design . New York: 
O tow Hill, 1962, p. 222. 
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In other respects, the treatment varied across the nine sites according to 
the requirements of local conditions. Given the diverse locations and 
circumstances in which a revised version of the ''hands off" ISTP might be 
used, the developers accepted wide variation in administration of the 
training program at the field test stage as an appropriate characteristic of 
the treatment- 

Evaluation Instrtmients and Test Procedures 

A specially constructed Instructional Supervision Test Battery served 
the purpose of generating scores for assessing the capability of the 
training program to meet its stated objectives. The Battery included: 

(1) a two-part Comprehension Exercise to test subjects' knowledge of 
central concepts presented in ISTP and subjects* ability to 
describe classroom behavior in observable, non-evaluative terms; 

(2) Observation Exercises Z and II to test subjects' ability to con- 
struct an observation instrument and to use the instrument system- 
atically in observing a classroom encounter; 

(3) an Analysis Exercise to test subjects' ability to analyze the 
results of systematically conducted observation; 

(4) a Participant Performance Record to assess the extent to which 
subjects appropriately utilized the approaches and skills included 
in ISTP when supervising a beginning teacher. 

The Comprehension Exercise, the Observation Exercises, and the Analysis 
Exercise were administered to Experimental Group I and Control Group I as 
both pretests and posttests. The exercises were administered to Experimental 
Group II and Control Group II as posttests only. The Participant Performance 
Record served as a post test only, administered at the conclusion of all 
training to experimental subjects only. Each training coordinator received 
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a full set of explicit instructions for administering the several exercises 
and the PPR to experimental and control subjects. 
Results of the Comprehension Exercise 

The first of the two parts of the exercise employed a 20-item multiple 
choice inventory to assess subjects' knowledge of a set of selected central 
concepts presented in the Readings and Handouts . The second part required 
subjects to classify correctly each of 35 descriptions of classroom behavior 
either as an observation or as a judgment, thereby providing a high infer- 
ence measure of their ability to describe teaching incidents in observable, 
non-evaluative terms. The Comprehension Exercise yielded two scores: the 
number of knowledge questions answered correctly; the frequency of items 
correctly identified as observations or judgments. 

The mean posttest scores for all four groups of subjects appear in 
Table 4. The experimental groups scored higher than the control groups on 
both measures. Training effects were statistically significant in each 
instance; a barely significant testing by training interaction appeared for 
the first measure. The data imply that training contributed materially to 
the ability of the experimental groups to identify correctly "knowledge" 
items and to distinguish between observational and judgmental statements. 

A comparison of the pretest and posttest scores of Experimental Group I 
only (Table 5) indicates that statistically significant increases occurred 
in both measures assessed by the Comprehension Exercise. The extent of the 
changes which occurred for the subjects in Experimental Group I offers 
additional confirmation to the inference that the training program contrib- 
uted significantly to increases in knowledge pertinent to ISP and to 
trainees' ability to distinguish between reports of observations and judg- 
ments • 
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TABLE S 



Pre and Post Mean Scores and Significance Test Results for 
Experimental Group I on the Comprehension Exercise 



Variable 



Pretest 



Posttest 



F-ratio 



1. No. of Correct Comprehension Items 



8*71 



16.93 



435.79* 



2. No. of Correct Observation/ 
Judgmental Items 



21.98 



31.03 



134.11* 



*p <.05;F(1,60)=4.00 

Results of Observation Exercises I and II 

In Observation Exercise I, the subjects received written background 
information about a teaching performance which they were soon to see. After 
they had completed their reading of the backgroimd information, the subjects 
viewed a two-minute video tape excerpt of the teaching behavior of a 
student teacher. They then were allowed approximately 10 minutes to plan 
their observations of a longer segment of the same video tape, with instruc- 
tions to concentrate solely on the questioning techniques us^ed by the 
student teacher. After the longer (nine-minute) video clip ran its course, 
subjects were permitted no more than 15 minutes to complete their observa- 
tional records. All subjects viewed the same video tape.* 

Observation Exercise I assessed the following training objectives: 
(1) developing a behavioral definition of an area of concern; (2) selecting, 
constructing, or adapting an instrument for recording observations; and, 
(3) making non-evaluative observations focused on the set of behaviors for 
which the observation instrument is specifically constructed. Behaviors 



*The elementary grade level teaching performance presented for observa- 
tion was selected from video tapes secured from the teacher preparation 
program. University of California at Santa Barbara. 
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pertinent to the first two objectives were scored for occurrence. Behavior 
pertinent to the third objective was scored for frequency of performance in 
two constituent categories: the total number of behavioral incidents (TBI) 
reported; the total number of judgmental statements (TJS) reported. The 
TBI score was the sum of (1) the number of discrete behavioral incidents 
recorded as specific non-judgmental statements (e.g., "The student teacher 
called on the boy in the red shirt.") and (2) the number of quantified 
behavioral statements recorded (e.g., "The student teacher called on the boy 
in the red shirt five times.") . The TJS score was the sum of (1) the number 
of discrete judgmental statements included in recording classroom events 
(e.g., "The student teacher was warm.") and (2) the number of quantified 
judgmental statements included in the recording (e.g., "The student teacher 
was never warm . 

The purpose of Observation Exercise II was to test the ability of each 
subject to construct, within 20 minutes, an observation scheme which he 
could use if he were to observe again the same video tape viewed in Observa- 
tion Exercise I. (The subjects did not actually employ the constructed 
scheme.) If the subject felt he had satisfactorily completed the assigned 
task during Observation Exercise I, he was asked to say so on his answer 
sheet and to submit the statement. The salient difference between the two 
exercises was that II addressed itself to the question of whether or not a 
subject could construct an appropriate observation instrument when specifically 
requested to do so whereas I addressed itself to the question of whether a 
subject actually did construct an instrument prior to conducting an observa- 
tion without receiving specific instructions to do so. If a subject con- 
structed an instrument in which not more than 25 per cent of the categories 
were judged by UCSB scorers to be non-behavioral in their specification, the 
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instrument was considered to be appropriately constructed. Similarly, if a 
subject designated the instrument he had actually used in Observation Exercise 
I as his response to Observation Exercise II, he was judged to have performed 
satisfactorily if not more than 25 per cent of the behaviors recorded fell 
outside the category system he had constructed. 

The mean scores for all four groups of subjects on the Observation Exer- 
cises appear in Table 6. On all six of the constituent measures, the scores 
for the experimental groups were significantly different, statistically, in 
the desired direction from the scores of the control groups (higher on Items 
1 - S and lower on Item 6). Table 6 also reveals statistically significant 
testing effects on Items 1 - S. Thus, the influence of previous test exper- 
ience on desired results cannot be discounted when comparing the posttest 
performances of Experimental Group I and Control Group I (the pretested 
groups). The extent of differences in the desired directions between Exper- 
imental Group II and Control Group II, however, attest to sizable training 
influence. 

With respect to pretest/posttest changes. Table 7 reveals that the 
subjects in Experimental Group I scored significantly higher, after training, 
on the same six measures identified in Tabl*^ 6. 

The test results reported in Tables 6 and 7, like the test results for 
the Comprehension Exercise, support the inference that the training program 
yielded desired changes in those specific behaviors which represented 
specific terminal objectives of ISTP. 

Reliability of ratings on Observation Exercises . Two expert raters, 
after reaching an agreed upon set of standards for each category, jointly 
rated the tests blind in two groupings: (1) pretests intermixed for experi- 
mental and control subjects, and (2) posttests intermixed for experimental 
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TABLE 7 

Mean Pretest and Posttest Scores and Significance Test Results 
for Experimental Group I on Observation Exercises I and II 





Measures for Observation 
Exercise I and II 


Pretest 


Posttest 


F-ratio 


1. 


% Behaviorally Specifying Area 
of Concern 


8.6 


96.55 


415.29* 


2. 


% Selecting Observation Instrument 


10.34 


96. 5E 


356.25* 


3. 


% Focusing on Specified Behavior 


3.45 


93.10 


494.00* 


4. 


% Constructing Observation Instrument 


22.41 


100.00 


197.30* 


5. 


Total No. of Behavioral Incidents (TBI) 


13.26 


59.78 


116.56* 


6. 


Total- No. of Judgmental Statements (TJS) 


9.69 


3.16 


50.34* 



*P ^.05;F(1,57) = 4.00 
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and control subjects. A test-ret est reliability on a random sample selected 
from both groupings of the tests yielded a mean Pearson correlation 
coefficient (r) of .99. Approximately three weeks elapsed between the 
test (original scoring) and retest (rescoring of random sample) for the 
posttest. The second scoring of the random sample of the pretest took 
place several months after the first scoring. 
Results of the Analysis Exercise 

The exercise provided scores for assessing the capability of ISTP to 
deliver the following analysis skills: (1) identifying a criterion for the 
successful performance of an intermediate goal; (2) constructing a data 
display; (3) making summary statements; (4) determining if a criterion for 
successful performance was met; (5) identifying patterns of behavior; (6) 
identifying antecedents and consequents of specific behavior; (7) specifying 
and justifying, in terms of anticipated pupil effects, which teaching 
behaviors are positive and which are negative; (8) identifying positive 
teaching behaviors which should be maintained; (9) identifying negative 
teaching behaviors which should be changed; (10) identifying alternative 
strategies for producing the desired changes. 

All subjects received an identical written protocol which contained a 
set of raw observation data and related background information. The observa 
tion records deliberately included examples of both patterns and individual 
acts of teaching behavior which would elicit application of the analysis 
skills to which training addressed itself. Subjects were requested to 
analyze the observation records according to instructions directed to each 
of the relevant skills. 
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Target abilities were scored for occiirrence, for increase in frequency 
of performance, or for both, as follows: 

fl) for occurrence of correct identification or determination identifying 
a criterion for the successful performance of an intermediate goal; 
constructing a data display; detennining whether a criterion for success- 
ful performance was met; 

(2) for frequency in number of instances identified — making summary state- 
ments; identifying patterns of behavior; identifying antecedents and 
consequents of specific bcliavior; specifying and justifying, in terms of 
anticipated pupil effects, which teaching behaviors are positive and 
which are negative; 

(3) for both occurrence and frequency — identifying positive teaching 
behaviors which should be maintained; identifying negative teaching 
behaviors which should be changed; identifying alternative strategies 
for producing the desired changes,* 

Table 8 presents the mean posttest scores for each group of subjects on 
the Analysis Exercise. A statistically significant training effect in the 
desired direction showed itself for 11 of the 17 constituent measures. On 
three measures (Items 11, 15, 16) the differences between experimental and 
control groups did not reach statistical significance but were in the 
desired direction. On two measures (Items 6 and 13), differences were in- 
consistent across the four subject groups. Item 2 showed no difference 



*In addition to scoring the subjects* responses for the number of 
instances where they identified positive and negative teaching behaviors on 
the grounds of specified student effects, scores were also tallied for the 
identification of positive and negative instances where no mentiom of 
possible student effect was made. The sum of the two scores provided a 
total number of identified teaching behaviors. 
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across groups because all subjects satisfactorily act the re<{uireaent of 
actuallx counting the particular behaviors which the/ had incorporated into 
their chosen criteria. 

A statisticallx significant testing effect was evident for Items 3« 
7, and IS; only for Iten 7 was there also a statistically significant 
testing by training interaction. The unusual behavior of Experinental 
Group II on Itea 7 deserves special coxansnt. The training prograa enphasited 
identification of both positive and negative incidents of teaching perfor- 
mance and presentation of justification for assigning positive or negative 
weight on the basis of possible effects on students. Inspection of Itens 
6 and 7 together rtcveals that both experinental groups xMde more identifica* 
tion statements tham both control groups, but for some unexplained reason 
Experimental Croup II offered KHAXiy more staten>ents without specifying 
reasons than any other group.* 

Table 9 shows the pretest and post test scores for Experimental Group 
I. On 15 of the 17 measures, statistically significant changes followed 
exposure to the training program. In the two instances wh; /e statistically 
significant differences did not appear, changes occurred in the desired 
direction. 

The data which appear in Tables 8 and 9 provide evidence that the train* 
ing program was successful in achieving oesireit results on the majority 
of the measures contaii/^ed in the Ajulysis Exercise. The significant testing 
effects associated with Items 5, *7\ and 15 could have occurred because of 
the very explicit nature of the instruction^ ysed ift the exercise. If 
experimental ^nd control grwips are used again for further revisic^ns of ISTP. 
it %^ld appear appropriate to modify the test instructions. 
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*Jiiote that lower scores are desirable for Itca 7. 
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TABLE 9 



Mfeta ^?retect aad Posttett Scores md Slgalflcance Test Results on 
AAsl/sis Exercise for Experimental Group I 





Tariablc 


Pretest 


Pott'tei*. 


r-ratio 


1. 


Bo. of 5aiia&ries 


6.06 


13.96 


60.31» 


2. 


of Subjects AaaJjnzlng Criterion 


66.66 


100.00 


6.6i»» 


5. 
J • 


Bo. of Psitt^ms 


1.51 


2.96 


6i*.0i** 


k. 


Bo. of A&t#c^dczitt/0ociseoi2eiits 


o.%6 


I.U3 


23.0<*» 




TV>t.sl Ho. of Stfttementfi of 
dusroon Behas;ior 


3.00 


U.U2 


l8.»»7» 


6. 


Total So. of Stait^emmts: 
Vlth Reason 


1.85 


3.62 


30.l8» 


7. 


Ibtal Mo. of Statesnents : 
Vitbout Reason 


1.00 


0.77 


0.67 


8- 


% of S<ti>Jects Specifying Cbsnges 


62.26 


90.57 


20.53« 


o. 


Bo. cf Changes 


1.15 


2.13 


2k. Bo* 


10. 


% of Subjects Specifying Maintenance 


26.li2 


66.0t» 


23.16« 


u. 


So. of Maintenance Bebariors 


0.32 


1.17 


2U.59* 


12 


C of Sub1#rt.s ST>«r*S ^irt ruf ^tratcfflss 




79.25 


16,56* 


13. 


Bo. of strategies 


1.13 


3.32 


M).2U« 




i of Subject? Speclfyliig Inter* 
apellate Coal 


U9.06 


88.66 


23.a6« 


15. 


i of Subjects Speclfytng at Least 
Obc Strategy for Each Change 


32.06 


81.13 


38.»*6« 


16. 


% of Subjects Specifying Change for 
Each negative 7eac.Mng Behavior 


37.7U 


56.60 


3.76 


17. 


% of Subjects ^peclfyiRg Maintenance 
for Each Positive Teaching Behavior 


11.30 


52.83 


28.15* 



•p£.05, r(i,53) - 1*.03 
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tolUbility of ratinis . Two expert raters Jjolntly rated the analysis 
test blindly in two different groupings: (1) pretest intermixed between 
cxperijiental and control subje<tts» and {Z) posttest intermixed between 
czperijiental and control subjects. A test*retest reliability on a random 
sample of combination of these t%#o groups yielded a sean Pearson 
correlation coefficient (r) of .99. A delay of approximately cne month 
oc^curred between the test (original scoring) and retest (rescoring of a 
radmon aampio) of the ppsttest scoring; several months elapsed between the 
two scorings of the pretest. 
Results of the Participant Performance Record 

Tbe Participant Perfon&ance Record (PPR) assessed the results of the 
overall training program on the subjects* performance in actual supervisory 
settings. Each supervisor received instructicns (1) to record a Pre- and 
a Post^servation with one of his supervisees in the school where the super* 
visee was teaching; (2) to observe the Supervisee's class using an observa- 
tloa instrument constructed for that purpose; (i) to analyte fully the 
recorded data with particular attention to identification of positive 
teaching behaviors to be maintained, to identification of teaching behaviors 
to be changed^ and to specification of strategies for achieving the 
desired changes. The supervisor's trio members evaluated the tapes and 
written materials for feedback purposed as the final activity in the trailing 
program prior to delivery of the protocols to UCSB for scoring* Each 
subject was extended the option of re-doing the entire fPti if he was not 
satisfied with his first effort. Fully coiqileted protocols were available 
for only S4 of the 76 experimental subjects* 
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Each teninml objective of the training progran vi5 scored according 
to the percmtage of prograa participants in the co&bined experinental 
groups vho successful 1/ performed Hue desired superviscyT/ behaviors in 
actTMl conferences with their supervisees. The ^scoring aUo noted ih^ether 
or not the trainees dcnonstrably evploxed the eight steps of ISP. 

T;Able 10 shows the percentage of S4 subjects who used the steps and who 
exhibited each of twenty-five constituent behaviors. Actually 27 separate 
scores appear in the table. Ho%^er, it$ns 5a and 8a represen^^ summary 
statements rather than indicators of constituent behaviors per se . Specifi- 
callyp item 5a says that 18.5 per cent of th^ trainees decided (with their 
supervisees) to set a criterion of performance. The proportions reported 
for behaviors 5ai and 5aii refer only to the 18.5 per cent and not to the 
entire trainee group. Item 8a says that all trainees did review the internes 
performance with the intern. The co\nstituent behaviors cited indicate how 
the trainees performed in those particular respects. Of the 25 behaviors 
scoreup the proportion of subjects who performed as desired was higher than 
the proportion who performed otherwise in 22 cases. In 17 of the 22 in- 
stances » the differences between the proportion \AiO did and the proportion 
who did n t exhibit the expected behaviors were statistically significajit . 
In the three instances where the proportion of trainees who did not perform 
as desired was greater than the proportion who did so (6e, 8aii, and 8c) « 
the differences were significant for the first (6e) and the third (8c). The 
scores for steps 5 and 4 were derived directly from the sole behavior sub- 
sumed (under each step and were, therefore, identical to the step itself. 

The scoring of the terminal objectives under Step 6 (Analyze the 
(K>servation Data) deserves special note. The two constituent behaviors 
%rhich all trainees were expected to exhibit were 6a (Construct data display) 
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TABLE 10 

Siamazy of Percentage of Participants Attaining 
Terminal OtO actives as Measured 
Participant Performance Record (PPR) 

THE INSTRUCTIONAL SUPERVISION PROCESS 



Terminal Objectives Yes No Not Applicable Chi-SQuare 



STEP 1 - 


Identify the area of 
concern in behavioral 
teres 


92.6 


7.1* 




yj.ko* 


la. 


To identify area of 
concern 


100,0 


0.0 




52.02* 


lb. 


To behaviorally define 
area of concern 


92.6 


l.U 




37. '♦O* 


STEP 2 - 


Decide to obtain a 
base rate or criterion 


87.0 


13.0 




28.17* 


2. 


To determine "whether 
the establishment of 
a base rate or the set- 
ting of a criterion is 
appropriate 


07.0 


13.0 


— 


28.17* 


3a. 


If criterion is appro- 
Iiriate » 


(18.5) 




(81.5) 






(i) to behaviorally 
define criterion 
(11) to set terminal 
goal 


100.0 
80.0 


0.0 
20.0 


— 


8.10* 
2.50 


3b. 


To identify intermediate 
goal 


5.5 


1.9 


92.5 


0.25 


STEP 3 - 


Select, r^odiiy or con- 
strucl^ en observation 
instrument 


100.0 


0.0 




52.02* 


k. 


To select, construct or 
adapt an instrument 
for the area of concern 


100.0 


0.0 




52.02* 


STEP k - 


Observe the behavior 
representing the area 
of concern 


100.0 


0.0 




52.05* 
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• p i. .05 

•• Xf«3.81*l, vith 1 d.f. at p».05 
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(Table 10, continued) 



Terminal Objectives 


Yes 


No 


Not Applicable 


Chi -Square 




5. 


To utilize the instru- 
ment to make ncn -evalua- 
tive observation in a 
classroom setting 


100.0 


0.0 


• 


52.05* 


STEP 


5 - 


Analyze the observa- 
tion data 


01. p 






20 .If 




6. 


To analyze the data 
ly performing the 
following operations: 














a. Construct data display 


98.1 


1.9 





48.17* 






b. Make summary state- 
ments 








40«± f 






c. Determine if cri- 
terion met if appro- 
priate 


oil n 

24.1 


3.7 


{2.2 


3.33 






d. Identify patterns of 
behavior 


55«o 






U .4 1 


ft 




e. Identify antecedents/ 
consequents of specific 
b^iaviors 


25.9 


7U.1 





11.58* 






f« S"DecifV "Dositive and 
negative behaviors, 
Justif^" in terms of 
anticipated pupil 
effects 


81.5 


18.5 





20.17* 


STEP 


6 - 


Identify behaviors need- 
ing maintenance or change 


57-.it 


U2.6 


www 


0.91 






To identify: 














a. positive behaviors to 
be maintained 


6l.l 


31.5 


7.U 


U.50* 






b. negative behaviors to 
be changed 


6Ji.8 


33.3 


1.9 


U.83* 






c. alternate strategies 


6l.l 


37.0 


1.9 


2.71 



for producing changes 
in intern tehaviors 
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(Sable 10, continued) 



Terminal O'bjectives Yes No Not Applicable Chi-Sguaxe 



STEP 7 " 


Provide feedback to 
Intern 




5.^ 




ltO.90* 


8. 


To perfonn the fol- 












lowing operations: 












a. Review vith intern: (100 .0) 


(0) 





(52.02*) 




(i) agreed upon area 


6U.8 


35.2 


— 


U.17* 




of concern and 












. its "behavioral 












definition 












(ii) criterion or base 


U2.6 


55.5 


1.9 


0.93 




rate predetermined 






37.'*0» 




(iii) the observation 


92.6 








instrument 












t. Guide intern in an- 


98.1 


1.9 




lt8.17* 




alyzing the data 












c. Assist intern to 


2U.1 


75.9 




13.50* 



identirty behaviors that 
are positive in terms 
of their possible pu- 
pil effects and should 
be maintained 



d. Assist intern to iden- 9h.k 3-7 1-9 UU.31* 

tify behaviors that are 
negative in terms of 
their possible pupil 
effect and should be 
changed 



STEP 8 - Develop strategies for 
behavior c±iange 


88.8 


9.3 


1.9 


33.23* 


e. Formiilate with intern 
strategies for achiev- 
ing the desired changes 


88.8 


9.3 


1.9 


33.23* 


f. Determine with the in- 


75.9 


2k.l 




13.50* 



tern if additional cy- 
cles of the Instruc- 
tional Supervision 
Process are necessary 
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and 6£ (Specify positive and negative behaviors with justification based 
on anticipated pupil effects). Otherwise, the UCSB staff did not expect 
each trainee would be able to, or necessarily should, perform all of the 
other four constituent analysis skills during the course of his Pre- and 
Post-Observation Conferences, on the grounds that the situation might not 
justifiably call for the employment of these particular skills. In view 
of the situational factors at work, PPR scores of the trainees on items 6c, 
6d, and 6e did not reach the same level as scores on other items in the 
area of Analysis. 

Reversal of expected behavior on 8c, as compared to performance on 8d, 
squares with previous experience in the UCSB training of supervisors. 
Apparently it is easier for supervisors to cite instances of negative 
behavior by student teachers and interns than to identify positive teaching 
behaviors. Also of interest is the difference between the written and 
verbal behavior of the trainees on items 8e and 8f . The scorers noted that 
while the subjects often did not put in writing their views on desired 
changes in supervisees* teaching performance, and associated strategies, a 
very large proportion were obviously prepared to discuss both the changes 
and strategies when they met the supervisee for verbal feedback sessions. 

The audio tapes of the Pre- and Post Observation Conferences also 
permitted the UCSB staff to score supervisor responses to supervisee as 
"freeing" or "binding". Table 11 summarizes the mean scores, standard 
deviations, and ranges for the two categories of responses. The scores 
reveal the participants used a far greater number of "freeing" than "binding" 
responses in conducting their conferences. 
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TABLE 11 

Sinnmary of Participant Scores in Demonstrating 
the Use of Freeing and Binding Responses 
Measured by the Participant Performance Record (PPR) 



Pre- § Post-Observation Conferences 


Mean Standard Deviations 


Ranges 


Total Freeing Response 
Total Binding Response 


39.70 30.21 
1.30 1.34 


4-150 
0*6 



Use of the steps of the Instructional Supervision Process . Table 12 
summarizes, in convenient fashion, the performance of the trainees on the 
eight steps of ISP. The pertinent behaviors were scored for occurrence on 
one or more occasions during the taped Pre- and Post-Observation Conferences. 
On seven of the eight steps of ISP, over 80 per cent of the participants 
performed the appropriate behaviors at high levels of statistical significance. 
The inability of the ISTP to deliver on Step 6, as compared to the other 
steps, is conspicuous. This datum, together with the data from two earlier 
stipervisor training efforts at UCSB confirms the common sense observation 
that analysis of the teaching act is a complex and confounding activity and 
pointed up the need for further conceptualization and implementation of the 
Analysis stage in revising ISTP. 

Table 13 indicates that approximately 78 percent of the trainees 
anployed at least seven of the eight steps of ISP in their live supervisory 
sessions. The mean number of steps completed by all participants for whom 
scores were available was 7.02. 

Taken together, the data contained in Tables 10, 11, 12, and 13 offer 
confirming evidence to analyses of the written exercises contained in the 
Instructional Supervision Test Battery that ISTP contributed substantially 
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TABLE 12 

Percentage of Participants Completing the Individual Steps of the 
Instructional Supervision Process and Significance Test ResTilts 



Instructional 
Supervision Process 


Yes 


No 


Not Applicable 


Chi -Square 


Step 1 


92.6 


7.U 


■ 


37.1tO* 


Step 2 


87.0 


13.0 




28.17* . 


Step 3 


100.0 


0.0 




52.02* 


St ep k 


100.0 


0.0 




52.02* 


Step 5 


81.5 


18.5 




20.17* 


Step 6 


57.1* 


U2.6 




0.91 


Step 7 


Sik.h - 


5.6 




U0.9O* 


Step 8 


88.8 


9.3 


1.9 


33.23* 



•X^ significant at p < .05 



TABLE 13 

Total Number of Steps of the Instructional Supervision Process and 
the Niamber and Percentage of Participants Completing Them 



No. of Steps Number of Participants Percentage 

8 2k hk.k 

7 18 33.3 

6 6 11.1 

5 3 5.6 

1» 2 3.7 

3 0 ' 0.0 

2 1 1.9 

10 0.0 

0 __0^ 0.0 

Total 5U* 100.055 



Mean No. of Steps 7.02 

*PER data were available for 3k of the 76 training participants 
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to the development of specified supervisoxy skills among subjects who received 
training. 

Reliability of ratings for the Participant Performance Record . The 
individual audio tapes were divided and scored by two expert raters- After 
the initial completion of the scoring, a random sample of sixteen tapes was 
selected and each rater scored that half of the tapes which he had not 
ixreviously rated. A Pearson correlation coefficient (r) of .99 was obtained 
between the ratings of the two scorers. 
Summary of Results of the Field Test 

The treatment consisted of administration of a "hands-off" Instructional 
Supervision Training Program in nine sites (eight Teacher Corps projects 
across the nation and one Santa Barbara group), involving 76 experimental 
and 31 control subjects, all volunteers- On-site coordinators who had no 
previous exposure to ISTP administered the training- The UCSB staff did not 
intervene directly in the training or in administering the Instructional 
Supervision Test Battery and associated evaluation instruments - 

An adaptation of the Solomon Four Group research design established 
conditions of pretest and posttest to two experimental groups and two 
control groups- The selected design assumes homogeneity of group composition. 
Because the UCSB staff was unable to insure random assignment of subjects at 
the several training sites, the composition of the groups was examined 
directly to ascertain the extent of similarities and differences- Chi- 
square analyses on 18 personal and experience characteristics revealed that on 
seven of the 18 characteristics statistically significant differences 
appeared in the composition of the four groups. 
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However^ when the entire experimental group was compared with the 
entire control group, a statistically significant difference showed itself 
on only Vmo characteristics: types of credential (s) possessed by the 
trainees and grade levels taught by the supervisee (s) supervised by the 
trainees . 

Pretest scores also permitted a degree of comparison between one experi- 
mental and one control group. No statistically significant differences 
appeared in the 26 summary measures incorporated in the Comprehension Exercise, 
in Observation Exercises I and II, and in the Analysis Exercise. 

While direct examination of selected personal characteristics of all 
subjects and comparison of pretest scores for selected experimental and 
control subgroups unearthed only a few salient differences, the assumption 
of homogeneity pertinent to the research design does not enjoy full support. 
The results of the field test should be interpreted in the light of the 
limitation unavoidably introduced because the UCSB Project Staff could not 
secure random assignment of subjects. Further caution in the interpreta- 
tion of results is pertinent because of the large differences in the sizes 
of the foui^ subject groups. 

The Comprehension, Observation, and Analysis Exercises provided one 
primary batch of posttest data for assessing training effects. On the 
Comprehension Exercise, both experimental groups scored at statistically 
significant higher levels than both control groups on the two measures 
included. On all six of the measures incorporated into the Observation 
Exercises mean scores for the experimental groups were significantly differ- 
ent, statistically, in the desired direction from the scores of the control 
groups. The influence of testing and interaction effects cannot be dis- 
counted in five of the statistically significant differences between Experi- 
mental Group I and Control Group I (the pretested gtoups). However, differ- 
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ences in the desired direction between Experinental Group II and Control 
Group II (not pretested) on each of the sane five itens attest to a sizable 
and consistent training influence. On the Analysis Exercise, a statistically 
significant training effect appeared for 11 of the 17 constituent measures. 
On three other nieasures the differences ran in the desired direction al* 
though they did not reach statistical significance. On three measures a 
statistically significant testing effect appeared » but there was only one 
instance of a statistically significant testing by training interaction* 

The second primary batch of data for assessing training effects case 
from Scores of the protocols included in the Participant Performance 
Record (PPR). Audio tapes and written materials from 54 of the 76 subjects 
in the two experimental groups were returned to UCSB for scoring the extent 
to which the trainees were abl*^ to perform the terminal objectives of ISTP 
within a live supervisory setting outside of the tiraining environment. Of 
25 behaviors scored, the proportion of subjects who supplied evidence of 
desired performances was higher in 22 cases than the proportion who did not 
perform as desired. In 17 of the 22 instances, the differences between the 
proportion who did and the proportion who did not exhibit the desired 
behaviors were statistically significant. In only three instances the 
differences in the undesired direction; two of these three differences 
were statistically significant. The PPR also provided evidence that the 
trainees used a far greater number of "freeing*^ than •*bfinding** re'i^ponses In 
conducting conferences with interns, a behavior pattern ^ich represented 
one of the 10 terminal objectives of ISTP. The PPR scores, finally^ 
revealed that over 44 percent of the 54 trainees for whc« full protocols 
were available used all eight steps in conjuction with their Pre-Observatlon 
and Post *Observat ion Conferences and that 78 percent Qf the trainees 
enployed at least seven of the eight steps. 
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Taken tofether» the data provided by the Cof^tehension Eicercise, 
^he Observation tjiercises» the toaiy^is Exercise^ and the Participant 
Performance Record offer substa/ttial evidence that the prelininary •'haiids- 
ofr^ trainint materials developed at UCSB contributed measurabl/ to improvinf 
the supervisory performances of subjects y^ho received training. All told» 60 
different scores %rere available for analysis of training effects; S3 of the 
a<eres were in the desired direction^ and 4S were statistically significant. 
Only two of the few scores wjhich w<^re in the undcsired direction were 
statisticaUy significant. 

TttOtKCK FROM UStRS 
Reactiofi from training sites to the employment of ISTP as an intervention 
strategy was primarily positive* and identified a number of specific points 
where revisions of the materials trere pertinent. Feedback came in the form 
of <}uestionaire responses from seven of the sites af>d of verbal input frcKB 
training coordinators who assembled for a Field Test Evaluation Conference in 
Santa. Barbara in April » 1973. The conferees agreed both that there was room 
for improvement in ISTP and that the materials did deliver training which was 
appropriate and helpful for the supervision of beginning teachers. According 
to verbal reports received at the conference^ trainees themselves had 
positively endorsed the training program. As a group* the assa^bled coordina* 
tors urged further revision of the ISTP package and distribution of the 
revised materials to all interested Teacliier Corps projects. They agreed 
also that ISTP was pertinent to use by supervisors in both teaci^er preparation 
programs and local educational agencies. As individual s» s^&veral coordina* 
tors sutoitted letters to the Teacher Corps program st^ff in the U.S. Office 
of Education in support of the UCSR project. 
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Sptcific positive coMtnts fros users included the observations thtt 
(1) the trainial prograa iras useful and relevant; (2) the Instructional Super- 
vision Process itself serreD as a w>del of foo4 teachinf technique; (5) the 
Instructional Supervision ftoct%% does ^free** beflnninf teadiers to male 
s«lf*evaluations; (4) ISTP 4ot% ^ress the inprovesent of classrooa instruct 
tion through tfce acquisition and application of pertinent supervisory skills; 
fS) ISTP should retain its focus on specific skills needed hy instructional 
supervisors in their %*ork with teachers on classrooa instructional behavior, 
even thoufh these skills represent only a limited portion of supervisory 
behavior broadly coiiceived. 

Users proposed a total of 2t specifi<^ revisions* IS in the Trainini 
Coordinator's Guide and 15 in the other components of ISTP,* P^f<o«»eudations 
for the Guide stressed (1) provision of more exan^ples of role pUyinf and 
observation instruments. (2) l»prove»ent of instnKtions for and substance 
in the area of Analysis. (3) clarification of the neaninf of the concepts of 
autmony and collef iality. (4) introd^oction of a "live^ practicun experience 
in lieu of a second rule playing practice exercise. (5) Improvement of 
instruction for choice of strategies by beginning teachers to improve their 
perfomance in selected areas of concern. (6) elimination of excessive 
directions in the Guide t^hen the directions could be placed in the Readings . 
(7) addition of bibliography on observational instn^sents. (t) addition of a 
glossary of key concepts and terms, and (9) provisiwi of »ore flexibility 
for the coordinator to incorporate optional activities int> the overall 
training program. 

Other reccumaendations for revision included (I) presenting the '"model** 
to depict more exactly the several stages incorporated in ISP. (?) tightening 



•See loyaa et, al. . 197$^ pp. 57-J9, for a full listing* 



tSm aaterials to cmmunicMtt wrc content in less space, (3) pifinatinf 

tk« Keadini s and KandoMts in an integrated forn, (4) addinf som large group 

activities to stLiauIate broader discussion opportunities for participants, 

(5) increasing the variet/ of illustrative concerns of beginning teacft^ers, 

(6) providing for switching trio Msibers over tine, (7) iaf)roving role placing 
exercises, (I) in^roving the substantive cotment and technical ^pialitx of 

tlie audio tape and video tape exeiqplars and stimili, and (9) iMlding a fwMb€T 
of descriptive transparencies for large group presentations. 

CQNCLUDtNC NOTg 

After anal/sis of field test results, rrvieur of questionnaire responses 
froB training sites, and ccmsideration of verbal assessments fron Teacher 
Corps trainers, each segment of the 1ST? materials ( Training Coordinator's 
Guide, leadings , Ha?>douts , audio tapes, and video tapes) ejcperienced 
extensive revisim. 

The revised package groups the eight sequential steps of instructional 
si^rvision into four stages: Pre^bservat ion Conferet>ce, Observation, 
Analysis, and Post -Observation Conference. The change in the representation 
of tt^ '""moder* of ISP will hopefully provide users with clear understanding 
of the relationship of the *^steps'^ to the ^stages*^ Hore specifically, the 
grouping of the **steps^ into *^stages^ should assist trainees to apply the 
constituent elements of ISP in systemically related fashion in real super* 
visory situations. (See Table 10 for identification of the steps and 
Figure 1 for representation of the stages.) 

The original seven instructional units have become 16, through re* 
constitution and rearrangement. Each of the revised units employs one of 
the following training modes: information transmission and acquisiticm. 



rolt pUxinf* or practiofli (tee Table 14). Estimated advaiitafes of the 
rMrrancettent are (1) that each instructional unit addresses itself to the 
developMtit of a specific skill and that the aaount of trait^ing tine for 
each unit is approxiaatelx the saae; (2) that trainees receive more oppor- 
tunities throughout the training for practicua ejperience5; (3) that, prior 
to each practicua experience, trainees d»onstrate their cotspetency with 
respect to knowledge about key concepts in instructional supervision and to 
skills in applying the concepts. 

The field test results pointed to the need for aajor revisions in the 
Analysis Stage of ISP, and associated training, despite salutary evidence 
secured froa the written Analysis Exercise. Data fron the PPR, especially 
scores for Steps S and 6, indicated that in a live supervisory situaticm, 
trainees required further assistance in aasts^^ng the Analysis domain. 
Training for the Analysis Stage of ISP (1) has dropped use of the notions 
of antecedent ax^ consequent events as strategic, (2) places heavier stress 
on the citation and maintenance (*f positive teacher behaviors while retain- 
ing cfl^iasis on eliminating negative behaviors, and (3) introduces the 
CBployi&ent of an entirely new Analysis protocol fom to insure inclusion 
by trainees of appropriate justifications related to probable effects on 
pupils in classifying a teaching behavior as positive or negative. 

Instruction for the Post-Observation Conference highlights (1) the 
essential need for supervisor review with supervisee of their initial agree- 
ments in the area of concern, (2) the importance of the supervisor's guiding 
the supervisee through analysis of the classroom observation data in a way 
irfiich approximates the analysis conducted separately by the supervisor, 
(3) the necessity for identifying instructional strategies which attend to 
the maintenance of positive behaviors as well as the elimination of 
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TABLE 14 



Instruct iofval Units of ISTP According to 
Instructional Mode 



Ic 

II. 
III. 

nr. 



VI. 
VII. 

VIII. 
IX. 
X. 
XI. 
XII. 
XIII. 

XIV. 

XV. 
XVI. 

o 

ERIC 



TITLES 

Introduction to Instructional Supervision 

Interpersonal Communication 

Behavioral Language 

Pre-Observation Conference: Step 1 - 
Behavioral ly Define the Area of Concern 

Pre-Observation Conference: Step 2 - 
Decide to Obtain a Base Rate or Set a 
Performance Criterion 



Pre-Observation Conference: 
Exercise 



Role Playing 



Pre-Observation Conference and Observa- 
tion: Steps 3 and 4 - Select and Use an 
Observation Instrument 

Pfe-Observation Conference and Observa- 
tion: Role Playing Exercise 

Pre-Observation Conference and Observa- 
tion: PractiCLun 

Analysis: Step 5 - Analyze the Observa- 
tion Results 

Analysis: Step 6 * Identify Behaviors 
Needing M^aintenance or Change 

Pre-Observation Conference, Observation 
and Analysis: Practicum 

Post-Observation Conference: Steps 7 
and 8 - Feedback the Data Results and 
Determine Strategies 

Analysis and Post-Observation Conference 
Role Playing Exercise 



Instructional Supervision Process: 
Playing Exercise 



Role 



Instructional Supervision Process: Prac- 
ticum 
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FOCUS 



Tnf ormat ion Ro 1 e 

Acquisition Playing Practicum 



negative behaviors, and (4) the value of insisting on following a decision 
to recycle the Process with observation of additionally specified super- 
visee behaviors. The reconstructed Post -Observation Conference Stage omits 
reference to discrimination training because training coordinators in the 
field test reported such training to contain little utility. 

The revised set of materials expand the capability for participant self 
evaluation. At the start of each training unit, the participant receives a 
statement of objectives to be mastered. During the course of the unit, 
presentation of lead questions enables the participant to check his progress 
toward mastering the stated objectives. At the end of each unit, the 
objectives appear again, for review purposes. 

Revision of ISTP occupied the UCSB staff during the summer months of 
1973. A fully revised training package, entitled the "Instructional Super- 
vision Training Program", was delivered to the U.S. Office of Education in 
September, 1973. 
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